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ABSTRACT 

We have carried out 870 /^m observations in the J1040. 7-1155 field, known to host an 
overdensity of Lyman break galaxies at z = 5.16±0.05. We do not detect any individual 
source at the S^o^m = 3.0 mJy/ beam (2 a) level. A stack of nine spectroscopically 
confirmed z > 5 galaxies also yields a non-detection, constraining the submillimctcr 
flux from a typical galaxy at this redshift to Sgroum < 0.85 mJy, which corresponds 
to a mass limit M(j ust < 1.2 x 10 8 M Q (2a). This limits the mass of thermal dust 
in distant Lyman break galaxies to less than one tenth of their typical stellar mass. 
We see no evidence for strong submillimeter galaxies associated with the ultraviolet- 
selected galaxy overdensity, but cannot rule out the presence of fainter, less massive 
sources. 
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1 INTRODUCTION 

Thermal dust, interstellar gas and stellar mass all contribute 
to the baryon budget of the universe. In the local uni- 
verse, all three components can be observed in great de- 
tail and the interplay between them studied. However, at 
high redshifts (z > 5) only the most luminous, rest-frame 
ultraviolet-selected stellar component has been subject to 
detailed scrutiny. 

Observations of the dust content of z > 5 g alaxies have 
been dominated by studies of quasar ho sts (e.g. IWang et al.l 
l200ct ) and submillimeter galaxies (e.g. ICoppin et alj|20ld l 
- relatively massive systems for their redshift and atypi- 
cal of the star-forming galaxy population. Individual stud- 
ies of far-infrared dust emission in more typical ultraviolet- 
luminous star-forming galaxies have bee n limited to unusu - 
ally luminous and lensed examples (e.g. iBaker et alj|200ll ). 
while photometrically-selected samples have been used to 
derive ensemble constraints from stacki ng analyses of UV - 
continuum selected samples at z ~ 3 (|Webb et all [2003) 
and galaxies selec ted for Lyman-cv emission at z ~ 5.7 
jCarilli et alj|2007h . 

Constraining the dust properties in these rest- 
ultraviolet selected populations as a function of redshift is 
key to understanding the early evolution of galaxies. The ul- 
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traviolet continuum is highly sensitive to the effects of dust 
extinction, leading to uncertainty in derived physical proper- 
ties. In particular, the clustering and star formation history 
of the universe at early times may be dramatically underes- 
timated if any significant fraction of the star-forming galaxy 
population is omitted from rest-ultraviolet selected samples. 

In our ESQ Remote Galaxy Survey (ERGS, 
iDouglas et al.l 12003 ). we have identified and studied 
the z w 5 'Lyman break galaxy' (LBG) population, selected 
for strong rest-frame ultraviolet continuum emission, in 
ten widely-separated fields. In two of these, J1040. 7-1155 
and J1054. 7-1245, we have detected evidence for large scale 
structure in the distant universe. This comprises overdensi- 
ties both in the number counts of a photometrically-selected 
sample and in the redshift distribution of spectroscopically 
confirmed galaxies, which shows a narrow (Az = 0.1) spike 
in source counts (Douglas et al, 2010, submitted). 

We have carried out 870 fim observations in the J 1040. 7- 
1155 field (which hosts a galaxy overdensity at Z — 5.16 ± 
0.05) using the Large Apex BOlometer CAmera (LABOCA) 
at the Atacama Pathfinder Experiment (APEX) telescope 
at Chajnantor in Chile in order to both probe the dust 
mass of the Lyman-break galaxy population and to test 
for the presence of ultraviolet-dark but submillimeter lu- 
minous galaxies that may form part of the same large scale 
structure. The unusual density of LBGs, and our extensive 
optical/near-infrared imaging and spectroscopy in this field, 
allows a simultaneous measurement of the cool dust con- 
tent of a large number of spectroscopically-confirmed z — 5 
galaxies. 
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In this paper we present a constraint on the dust mass 
of z = 5 LBGs. Throughout, we adopt a ACDM cosmology 
with (Ov, Qm, h)=(0.7, 0.3, 0.7). 



2 OBSERVATIONS 

Observations were carried out using the Large APEX 
BOlometer CAmera at APEX, associated with ESO pro- 
gramme 083.A-0199 (PI Stanway). A total of 11.7 hrs of 
data were collected in service mode over the period 2009 
May 10th - 2009 July 29th, of which 9 hrs were spent on 
the target field. Data were taken using the standard spi- 
ral mapping pattern on a four-point raster to increase the 
area surveyed, and skydip observations were used to correct 
for atmospheric opacity. Precipitable Water Vapour (PWV) 
measurements at the site were typically 0.4-0.7 mm during 
observations. Pointing was checked once an hour. Standard 
sources CW-LEO, B13134 and N2071IR were used for point- 
ing and flux calibration. Flux uncertainty was typically 3%. 

The field was centered at 10 h 40 m 42!2-ll o 58'00" 
(J2000), on a z = 5.12 galaxy identi fied from its CO(2-l ) 
emission in the rest-frame far-infrared (jStanwav et al.l l2008). 
Discussion of this non-LBG source is deferred to a later pa- 
per. Unfortunately the project was not completed before the 
end of the semester, leading to an irregular coverage of the 
field, with regions of the survey area (which partly overlaps 
our optical imaging) significantly deeper than others. A to- 
tal area of approximately 9.' 7 x 9.' 7 was surveyed to 50%, 
and 7.'7 x 7.'7 to 75%, of the best total coverage (figure [TJ. 

Data were reduced using the dedicated Bolometer Ar- 
ray (BoA) analysis software developed to reduce LABOCA 
data, using an algorithm optimised to recover weak sources. 
LABOCA has a beam size of 18" on the APEX telescope 
and the data were resampled to a pixel scale of 9" per pixel 
during the reduction proced ure. In common with other au- 
thors (e.g. IWeifi et alj|2009h . we smooth our data by con- 
volution with the beam width in order to optimise signal 
to noise for point sources. The root mean square noise on 
the smoothed image was 1.49mJy/beam (measured over a 
region with mean coverage 84%). 



3 A LIMIT ON THE DUST MASSES 

Twelve spectroscopically-confirmed z > 5 Lyman break 
galaxies lie within the field of our LABOCA observations 
(shown by circles in figure [T]). Of these, two lie southwards 
of the field centre in a region with poor flux sensitivity and 
are not considered further in this analysis. Two other sources 
are separated by a projected angular distance of only a quar- 
ter beam although there is actually a substantial line of sight 
distance between them, with one source lying at Z — 5.15 
and the second being a background source at z — 6.41 (orig- 
inally selected as an /-band dropout galaxy). This is the sole 
outlier in our spectroscopically confirmed sample, with the 
remaining galaxies lying in the redshift range 5.1 < z < 5.5, 
with a mean redshift < z >= 5.177. 

None of the confirmed z — 5 galaxies in this field are 
individually detected. Indeed, there is, at most, one con- 
vincing 3-5 a detection across our field (lying well north of 
the region surveyed in our optical imaging and consistent 



with the 1-2 sources expected based on o ther 850 /im sur- 
veys, [CoiElneiaD[200i; IwifieEaQ Eool) . The 18" beam 
of LABOCA corresponds to an angular scale of 112 kpc at 
2 = 5.18, and the stellar light of Lyman break galaxies in our 
sampl e is typically extend ed over a half-light radius of just 
0.14" (|Douglas et al J 120091 ) . We thus expect any dust emis- 
sion from our distant sources to be unresolved in LABOCA 
imaging, and can place a 2 a upper limit Ss70pm < 3.0 mjy 
on the flux from each galaxy. 

We also consider a mean stacked image constructed 
from a 40 x 40 pixel (20 x 20 beamsize) region surround- 
ing each of the nine z ~ 5 LBGs labelled in figure [T] Eight 
of these show Lyman-a emission lines, although only four 
have line equivalent widths > 30A, sufficiently high for de- 
tection as Lyman-alpha emitters (LAEs) in typical narrow- 
band surveys. Since our nine sources lie in a region with 
near-constant coverage we use a simple mean; using a noise- 
weighted average changes the results by <2%, well within 
the statistical errors. An inspection of this stack also failed 
to yield a detection, with an improved upper limit on the av- 
erage source of SsrO/aa < 0.85 mjy (2 a, based on the image 
noise, which is well fit by a Gaussian distribution). We note 
that contributions from sources along the line of sight be- 
tween us and the z = 5 target galaxies or from the z — 6.4 
background source can only increase the measured flux in 
the stack; hence this is a strict upper limit. Emission from 
line-of-sight sources offset from the targets will increase the 
noise levels in the image stack and such confusion may be 
a limiting factor on our point-source sensitivity. Again this 
effect acts to increase rather than decrease our upper limit. 

Following I Aravena et al.l (2008, cqns 3 & 4), we convert 
our flux limit into a constraint on the dust mass of each 
galaxy by modelling the spectrum as a grey body, taking 
into account dust heating by the CMB as a function of red- 
shift and observing frequency (a significant effect at high 
redshifts). We do not assume an optically-thin approxima- 
tion but use the full expression. In the absence of quan- 
titative information about the properties of dust grains at 
very high redshift we us e the same far- i nfrare d dust emis- 
sivity coefficient used by I Aravena et al.l (|2008l ). The power 
law emissivity inde x is fixed to 8 = 2, found to fit a sample 
of z > 4 quasars bv lPriddev fc McM ahon (2001) and rather 
steeper than the B — 1.6 found for lower redshift quasars by 
iBeelen et al.l (|2006T ) . This steeper e missivity law i s appropri- 
ate for relatively small dust grains (|Desert et alJll99CT ) and 
hence may arise from ev olution in the dust p roperties of 
galaxies at high redshift. iMaiolino et al.l (J2004J) found that 
rest-frame ultraviolet dust absorption in z > 6 sources is 
best described by smaller grains with production dominated 
by Type-II S Ne, an effect also seen in z > 5 gamma r ay burst 
host galaxies (|Stratta et al.ll2007l ; [Perlev et alj|2009h . Alter- 
ing the emissivity index from B — 2 to 8 = 1.6 increases the 
derived dust masses by a factor of 2. 

Figure [2] shows our constraint on the dust mass of a 
typical galaxy in our sample as a function of assumed dust 
temperature. The LABOCA non-detection limit from our 
LBG stack either constrains the dust temperature in a typ- 
ical UV- luminous galaxy at z — 5.18 to < 30 K or the dust 
mass to Mdust < 1.2 x 10 8 Mq (2<j), with constraints on 
individual galaxies being a factor of three times weaker. 

The highly lensed z — 2.7 Lyman break galaxy MS1512- 
cB58 has a dust temperature T= 33 K based on fitting of 
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Figure 1. Our LABOCA map (left, smoothed by convolution with the 19" beam) and associated coverage map(right). The dashed line 
denotes the region within which we have existing optical data and circles mark the optical locations of spectroscopically-confirmed z ~ 5 
Lyman break galaxies. Each circle matches the beam FWHM for an unresolved source. Numbered circles indicate sources used in the 
stacking analysis. Colour is inverted so black indicates a high-flux signal. 



its su bmillimeter spectral energy distribution (|Baker et al.l 
I2001T ). The mean dust temperature of submi llimeter selected 
galaxies at z ~ 2 is com parable: ~ 35 K (|Chapman et al.l 
l2005l ; lKovacs et alj |2006). Our galaxies are less massive, and 
their integrated star formation rates lower, than typical sub- 
millimeter galaxies. However, the star formation intensity 
in their central regions is comparable - a few tens of sola r 
masses per year per square kiloparsec IjVerma et alj|2007h . 
Hence we adopt 30 K as an estimate for dust temperature 
(and the resulting mass constraint) in the discussion that 
follows. 

At this temperature, assuming the emissivity index and 
dust model discussed above, Lfir = 2.8 x 10 12 SWo^m for 
a source at Z — 5.16, where Lfir is the integrated far- 
infrared luminosity integrated between 40 and 1 20 /im in 
the re st-frame and measured in solar luminosities ICondonl 
|l992j) and Sg7o^ m is the observed flux at 870 /im in Jan- 
skys. Hence our flux limit implies a typical z ~ 5 LBGs has 
Lfir < 2.4 x 10 9 I/©. We do not extrapolate further to obtain 
a star formation rate from this value, since the assumptions 
requir ed to do so at z ~ 5 are far from clear. As lCarilli et al.l 
(2008) discuss, the far- infrared to radio luminosity corre- 
lation is expected to break down at high redshifts, where 
inverse Compton scattering off the cosmic microwave back- 
ground is likely to be a significant effect. 



4 DISCUSSION 

4.1 LBGs atz~5 

This work provides the first constraint on thermal dust emis- 
sion from spectroscopically-confirmed, rest-ult raviolet con- 
tinuum selected galaxies at z ~ 5. However, ICarilli et al.l 
(J2007j) derived an upper limit on the 1.2 mm flux of a sam- 
ple of ten galaxies selected to have a flux excess consistent 
with strong Lyman-cv line emission at z ~ 5.7. Applying 



the same analysis used here and assuming T= 30 K, their 
stacking analysis limit on the typical flux of these sources, 
Si. 2mm < 0.7 mjy (2 a), corresponds to a dust mass limit 
Mdust < 1.9 x 10 8 Mq. Galaxies selected as LBGs are typi- 
cally more luminous in the rest-frame ultraviolet continuum 
and hence believed to be older and more massive than sim- 
ilar sources selected fo r a very high L yman- a emission line 
equivalent width (e.g. lLai et al.l 12008). However, the dust- 
mass constraint in this paper is now within 30% of the tight 
limit on dust mass in z = 5.7 Lyman-a emitters, suggesting 
that neither of these populations hosts any appreciable dust 
component. 

The stellar masses of L yman break galaxies at z ~ 5 
are ty pically ~ few x 10 9 M Q |Verma et al.ll2007l ; lst"ark et all 
120091 ). Our limit on the dust mass of these galaxies Mdust < 
1.2 x 10 8 Mq at 30 K constrains their baryonic mass compo- 
nent in the form of thermal dust to be no more than about 
a tenth of the stellar component. 

Interestingly, this upper limit is already comparable to 
the stellar mass to warm dust ratio observed in starbursts in 
the local universe. iHunt. Bianchi. fc Maiolind (J2005J) stud- 
ied dust emission from a small sample nearby blue com- 
pact dwarfs, which are comparable in both metallicity and 
star formation intensity to z ~ 5 LBGs, albeit less massive, 
and have emission dominated by small-grained, supernova- 
formed dust muc h like galaxies at z > 5. Fro m SED fitting 
of the starbursts, IHunt. Bianchi. fe Maiolind derived a stel- 
lar mass to dust mass ratio of 10 and 13 for two sources 
with Z/Z0=O.14 and 0.20 respectively. The metallicity of 
Z ~ 5 LBGs is still poorly constrained, although estimates 
in t he range 0.1-0.2 Zq appear appropriate from SED fitting 
(e.g. IVerma et al.ll2007h and analysis of the rest-UV spectral 
slope (Douglas et al, 2010). 

Lyman break galaxies at z ~ 3 are selected for rest- 
frame ultraviolet continuum in the same manner as the z ~ 5 
LBG sample discussed here, and hence have comparable star 
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Figure 2. Constraints on the dust mass derived from our 870 itm 
flux limit for a stack of nine z = 5 Lyman break galaxies. Contours 
show dust masses excluded at 1, 2, 3, 4 and 5 a level as a function 
of black body temperature. 



formation rates, and yet differ significantly in their physical 
properties. Lyman break galaxies at z ~ 3 are typically older 
and more massive than those at z ~ 5 (having had longer to 
evolve). They have similar or slightly higher metallicities and 
also appear to hav e slightly larger dus t extinction in the rest- 
frame ultraviolet IjVerma et al J 120071 1. Despite these differ- 
ences, the well-studied z ~ 3 population provides the most 
direct comparison set with UV-continuum selected samples 
at higher redshift. 

Individual detections of thermal dust emission have now 
been made in several z ~ 3 star-forming galaxies, gener- 
ally made possible by a high degree of gravitational lens- 
ing. The well-known z = 2.7 source MS1512-cB58 hosts 
a dust mass of 1.2 x 10 7 Mq (based on the 1.2 mm flux 
of iBaker et"afl l200ll . using T=30K, = 2 and correct- 
ing for lensing /x = 32) and a stellar mass of ~ 10 9 M 
l|Siana et alj200a 1. Similarly the z = 3.07 Cosmic Eye, hosts 
a stellar mass M* = 6 x 10 9 Mq and an estimat ed dust 
mass Mdust = 2.5 x 10 7 M Q l|Coppin et alj|2007h . These 
very low dust masses are consistent with an ensemble av- 
erage estimated from a stack of submillimeter fluxes for 
photometrically-selected z ~ 3 Lyman break g alaxies from 
the C anada-UK Deep Submillimeter Survey. IWebb et al.l 
(|2003r > measured a 2<r upper limit on the flux of a typi- 
cal z ~ 3 LBG, Ss50p,m < 0.4 mjy, corresponding to a dust 
mass Mdu S t < 5 x 10 7 Mq at 30 K using the cosmology in 
this paper. 

If the relatively low dust masses seen in z ~ 3 LBGs are 
common at higher redshifts, then significantly deeper sub- 
millimeter data will be required to detect the dust emission 
from z ~ 5 sources. However our limit is already sufficient 
to challenge any paradigm in which a significant amount of 
the star formation at z ~ 5 is heavily obscured. 

This argues against models in which the multi- 



component morphology of many rest-ultraviolet selected 
high reshift galaxies is analogous to the distribution of 
massive starburst regions seen in local ultraviolet-luminous 
galaxies, while the bulk of the underlying galaxy remains un- 
seen due to heavy obscuration. The high far-infrared fluxes 
predicted for such embedded obscured clumps are hard to 
reconcile with the tight subm illimeter constraints emerg ing 
for high redshift populations IjOverzier et alj|200a . 12009 1. 



4.2 Structure in the J1040. 7-1155 Field 

While the photometric properties of individual Lyman break 
galaxies contributing to our analysis are rather typical of 
z ~ 5 Lyman break galaxies as a whole (Douglas et al, 2010), 
it is worth noting that the large scale structure properties 
of the field are far from typical. The redshift distribution 
of spectroscopically confirmed galaxies in the J1040. 7-1155 
field shows a 6 a peak relative to the mean distribution in our 
spectroscopic survey, lying at z = 5.16±0.05. The field is also 
overdense in photometrically selected z ~ 5 candidates and 
hosts an ultraviolet-dark z — 5.12 galaxy with a large mass 
of gas in the form of carbon monoxide (and, by inference, 
molecular hydrogen. IStanwav et alj|200a . 2010 in prep). Six 
of the galaxies contributing to this analysis form part of the 
large scale structure in this field. 

Two of these sources, objects 3 and 5 in figure [l] have 
tight constraints on their molecular gas masses: M(H2)< 
1.7xl0 10 andM(H 2 )< 2.9xlO 9 M (2 a) respectively, based 
on measurement s of carbon m onoxide emission at millime- 
ter wavelengths 1| Stanway et a l. 2008). Object 3 is confused 
with a bright neighbouring source in our deep Spitzer/IRAC 
imaging of this field, but otherwise typical of Lyman break 
galaxies in our sample. In the case of object 5 we are also 
able to constrain its rest-frame optical flux based on non- 
detection in our IRAC imaging and estimate its stellar mass 
M* ~ 2.5 x 10 9 (Stanway et al, in prep). In this source, 
a confirmed z = 5.116 galaxy, the stellar mass contributes 
a minimum of half of the observed baryonic content of the 
galaxy, with the mass in molecular gas potentially compa- 
rable to this, and a contribution from thermal dust < 10% 
of the observed total. 

These data allow us not only to study the properties 
of known rest-frame ultraviolet luminous sources, but also 
to determine whether massive, dusty sources exist within 
the same large scale structure at very early times. The ex- 
cess of Lyman-break selected galaxies may well be trac- 
ing out massive large scale structures in which the ma- 
jori ty of baryonic mate rial is undetected in the ultravio- 
let (jStanwav et alj|200a 1. If massive submillimeter galaxies 
preferenti ally occupy strong peaks in the matter density dis- 
tribution (jBlain et al.l 120041 ). we might expect to detect an 
excess such galaxies in this field. Such be haviou r has been 
observed using LABOCA bv lBeelen et al.l (|2008I 1 who iden- 
tified a possible overdensity of submillimeter galaxies ap- 
parently associated with strong Lyman -a emitting s ources 
forming a protocluster at z — 2.4, by llvison et al.l (2000) 
who observed massive SCUBA galaxies clustering with Ly- 
man br eak galaxies in the env irons of a z — 3.8 radio galaxy, 
and by lAravena et al.l (|2010l 1 who observed BzK galaxies 
clustering with MAMBO-selected sources at z ~ 2. Given 
the current lack of information on typical luminous lifetimes, 
dust properties, clustering scales and mass scales of z ~ 5 
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LBGs, submillimeter and gas-dominated galaxies, it is im- 
possible to make a quantitative prediction for the number of 
submillimeter sources expected in an overdensity of Lyman 
break galaxies. However there is no evidence for this field 
being overdense in submillimeter galaxies at any redshift. 

At dust temperatures of ~30-35 K the lack of 3 a detec- 
tions for continuum sources in our field implies a dust mass 
limit of ~ 2 — 4 x 10 8 Mq at z ~ 5, ruling out the pres- 
ence of a massive UV-obscured active galaxy or submillime- 
ter galaxy co mparable to others seen at high redshift (see 
compilation in lSolomon fe Vanden Boutll2005r ). However, we 
caution that our constraints on individual sources are weak 
(and depend on the assumed dust temperature). We cannot 
rule out a substantial population of cooler, less massive or 
less dusty sources in this field. The apparent lack of massive 
evolved systems at this redshift may indicate that galaxies, 
even those in apparently overdense regions, may have a more 
complex history than simply growing into massive galaxies 
observed locally. 



5 CONCLUSIONS 

We describe 870 /im observations undertaken using 
LABOCA and targetting a field known to host an overden- 
sity of Lyman break galaxies at high redshift. Our main 
conclusions can be summarised as follows: 

(i) No individual galaxy is detected in our LABOCA ob- 
servations, to a a 2 a upper limit SWo^m < 3.0 mjy. 

(ii) By stacking nine spectroscopically confirmed z ~ 5.2 
galaxies in this field, we are able to constrain the thermal 
dust emission in a typical Lyman break galaxy at this red- 
shift to Ss70iim < 0.85 mjy, corresponding to a dust mass 
M dust < 1.2 x 10 8 M (2a) at a dust temperature of 30 K. 

(iii) These observations place an upper limit on the ther- 
mal dust fraction in a typical LBG at z ~ 5 of ~ 10% of the 
stellar mass. 

(iv) We see no evidence for an excess of heavily-obscured 
massive starburst galaxies associated with a large scale 
structure at z — 5.16 in this field. 

Deeper submillimeter observations are required to bet- 
ter constrain the dust properties of galaxies at high redshift. 
Our observational programme to characterise z ~ 5 galax- 
ies is continuing, with particular emphasis on both this field 
and a second overdense region. We note that the Atacama 
Large Millimeter Array (ALMA) will have a transformative 
effect on these studies, making dust continuum detections 
in typical, unlensed Lyman break galaxies accessible for the 
first time. However the small field of view of ALMA at sub- 
millimeter wavelengths (just 13arcseconds at 250 GHz) will 
reduce its effectiveness for large scale mapping of the rela- 
tively sparse high redshift galaxy population. 
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